Light Field Camera
Motivation
We found that capturing a large set of images over a plane perpendicular to the optical axis enables complex post-capture refocusing effects. We aim to reproduce these effects by simply shifting and averaging multiple sub-aperture images from a light field dataset.
Depth Refocusing
When capturing a light field, we take photos from multiple viewpoints without changing the optical axis. Objects closer to the camera show significant positional shifts across these images, while distant objects remain relatively still. Averaging all images as-is will emphasize distant objects (in focus) and blur nearer objects. By shifting each sub-aperture image toward a central view before averaging, we can "refocus" the scene at different depths. Varying this shift parameter (C) changes the focal plane of the resulting image.
Below are examples showing depth refocusing for various C values. Each sequence transitions from C = -3 to C = 3 in increments of 0.3, illustrating how the focus plane moves through the scene.
Aperture Adjustment
By averaging a subset of the images around the central viewpoint, we can simulate changing the aperture size. Using fewer images corresponds to a smaller aperture, yielding a sharper overall image, while using more images simulates a larger aperture that narrows the depth of field.
Below are examples showing aperture adjustments as we vary the number of sub-aperture images included (from 0 up to 50). For the chess dataset, we used C=0.4; for the lego dataset, we used C=0.3.
Summary
Through simple operations like shifting and averaging a grid of images, we recreated the key optical properties of a light field camera: refocusing at different depths and simulating various aperture sizes. This demonstrates that the light field data structure holds rich information that can be leveraged for post-capture manipulations without complex, computationally expensive algorithms.
Gradient Domain Fusion
Motivation
In Project 2, we blended images using Laplacian and Gaussian pyramids but struggled to create perfect masks. To overcome this, we explore Poisson Blending, which focuses on gradients and naturally integrates source content into the target image without relying on carefully crafted masks.
2.1 Toy Problem
Before implementing Poisson Blending, we first solve a simpler problem: reconstructing an image from its gradients and a single pixel constraint. We represent the image as a vector and formulate a system of equations Av = b, where A encodes the gradient constraints (both x and y directions) and one intensity constraint. By solving this system, we recreate an image nearly identical to the original.
Below are the original toy image and the reconstructed version. The maximum error between them is very small, demonstrating that our gradient-based system can accurately reconstruct the image.
Poisson Blending
Building on the toy problem, we now set up a similar system of equations for blending a source image into a target image. Poisson Blending focuses on preserving the source gradients inside the defined mask region while ensuring that pixels at the boundary align smoothly with the target image. By solving Av = b for each color channel, we seamlessly integrate the source content into the target background.
Mixed Gradients
Mixed Gradient blending extends Poisson Blending by choosing gradients from either the source or the target image based on which has a larger magnitude, often producing visually appealing composites even when source and target images differ significantly in texture and intensity.
